Skip to content

test: gate operational self-knowledge contradictions - #23

Merged
ShuhaoZhangTony merged 1 commit into
mainfrom
codex/issue-12-runtime-contradiction-gate
Aug 15, 2026
Merged

test: gate operational self-knowledge contradictions#23
ShuhaoZhangTony merged 1 commit into
mainfrom
codex/issue-12-runtime-contradiction-gate

Conversation

@ShuhaoZhangTony

Copy link
Copy Markdown
Contributor

Summary

  • derive operational expected facts from /health or an independent fixture instead of encoding one model name
  • evaluate 24 Chinese/English, typo, follow-up, misleading-premise, and stack-collaboration questions
  • fail on missing required facts, forbidden runtime claims, used_model mismatch, missing Support, or wrong decision route
  • record trace, stage timings, citation coverage, and contradiction score in a JSON artifact
  • make the semantic gate part of hosted deployment verification
  • expose structured speculative non-activation reason and platform-aware SAGE/vLLM-HUST/backend roles

Validation

  • uv run ruff check src tests tools/validate_operational_self_knowledge.py tools/verify_hosted_web_deploy.py
  • uv run pytest -q — 721 passed, 28 skipped
  • focused operational/runtime/receipt suite — 78 passed
  • isolated HTTP gate — 24/24 passed, reference coverage 1.0, max contradiction score 0.0

Closes #12

@ShuhaoZhangTony
ShuhaoZhangTony merged commit 5354be6 into main Aug 15, 2026
5 checks passed
@ShuhaoZhangTony
ShuhaoZhangTony deleted the codex/issue-12-runtime-contradiction-gate branch August 15, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1][Regression] Add operational self-knowledge contradiction and evidence gates

1 participant